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BACKGROUND 



This report addresses the maintenance phase of the software life cycle 
and the methodologies and procedures which, if employed during the soft- 
ware development and design phases, will improve software maintainability. 
We define maintenance as that activity which is concerned with making 
changes to software for the purpose of improving or correcting the soft- 
ware. Maintainability is a property of the software which makes the 
maintenance activity easy to perform, i.e., changes to the software are 
easy to incorporate and do not lead to new errors in the software. 

The Trident Command and Control Systems Maintenance Agency (CCSMA) 
tasked the Naval Postgraduate School to study and evaluate various Navy 
software standards with regard to their applicability to software 
maintenance. The question posed was: Could these standards, accompanied 
by basic program documentation, such as a listing, provide adequate 
guidance for a new programmer to maintain software, such as that found 
in the Trident Command and Control Subsystem? Related to this question 
is the critical relationship between development and maintenance. For 
the most part, the standards and documentation techniques which were 
reviewed (e.g., MIL-STD 1679) were designed to be used for software 
development and not for maintenance, specifically. This situation illus- 
trates the fundamental problem in software maintenance : inadequate 

attention to this important and high cost phase of the software life 
cycle. Since the conditions for future maintenance problems are created 
during development and design, the approach taken in this study was to 
look to these phases of the life cycle as the areas having the greatest 
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potential for improving the maintainability of software. This concept 
suggests that maintainability must be designed into the software and 
that development methodology has a more significant effect on main- 
tainability than maintenance practices themselves. However, pre- 
maintenance phase activities should not be emphasized to the exclusion 
of maintenance phase activities. Maintenance practices, such as patch- 
ing programs and deficiencies in testing subsequent to a change (e.g., 
lack of regression testing) , obviously exert a deleterious effect on 
the quality of software. Unfortunately improvements in maintenance 
practices cannot cure underlying problems which are introduced into the 
software when it is specified and designed. Hence it is clear that 
significant gains in maintainability can only be achieved by recog- 
nizing maintainability objectives as an integral part of development 
and design. 

Consistent with the above problem approach, this report begins with 
a discussion of the findings of a number of researchers, in the areas of 
software development and maintenance practices, which bear on the problem 
of improving software maintainability. This is followed by an evalua- 
tion of Navy weapon system software standards (WS 8506 and MIL-STD 1679) , 
from the standpoint of their effectiveness as aids to the maintenance 
programmer. 
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I . INTRODUCTION 



Software maintenance is one of the most expensive phases of the 
system life cycle. One author suggests it is as high as 67 per cent of 
the effort in large-scale systems [l]. Despite this fact, maintenance 
requirements receive little attention during system development. There 
are many reasons for this situation, but three stand out: 

1. Maintenance is considered less glamorous, interesting and 
challenging as compared to system design and programming; hence, there 
is little incentive for computer personnel to become involved in main- 
tenance activities . 

2. During development it is often too early in the project to 
foresee problems which may occur during maintenance; as a consequence 
maintainability is not provided in the system design. 

3. Project management does not always recognize that maintainabil- 
ity considerations should be an integral part of the design process. 

One result of this lack of attention to maintainability require- 
ments during design is loss of tre ace ability. This is defined as the 
ability to identify the technical information which pertains to a 
software error which has been detected during the maintenance phase 
and thereby trace the error to the applicable design specifications 
and user requirements statements. The same traceability capability 
is also required for software improvements which are made during the 
maintenance phase. It is clear that if significant improvements are 
to be made in software maintainability, requirements for maintenance 
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must be made an integral part of the software specification develop- 
ment and design process • In fact, as mentioned later, some software 
developers recommend that software design be oriented around the need to 
maintain the software in its operating environment. 
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II. OBJECTIVES 



The objectives of this report are: 

1. In particular, to evaluate Weapons Specification (WS) 8506 [2] 
and Military Standard (MIL-STD) 1679 [ 3 ] with regard to their suitability 
for computer program maintenance in embedded computer systems. 

2. In general, to discuss and propose software development prac- 
tices which will lead to improved maintainability for any software 
system. 

In 1. emphasis will be on the evaluation of the ability of these 
documents to provide for traceability during program maintenance. 
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III. APPROACH 



The WS 8506 and MIL-STD 1679 documents were examined with regard to 
their effectiveness for software maintenance purposes — in particular , 
their capability for providing traceability. In addition to this specific 
aspect , all sections of these documents which are applicable to maintenance 
were reviewed, and strong and weak points were noted. A major approach 
in this review was the use of a rich body of literature in software 
engineering which was employed as an information source for suggesting 
improvements in the documents. This literature and our concept of the 
characteristics of good specifications and standards for maintenance 
were used to develop a set of criteria for judging WS 8506 and MIL-STD 
1679. 
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IV. GOOD MAINTENANCE PRACTICES 



A. Design Approaches for Achieving Good Maintainability 

As described in Yourdon [4], a good software design and maintain- 
ability strategy is that of divide and conquer. The cost of implemen- 
tation will be minimized when parts of the problem are manageably small 
and separately solvable. Similarly, the cost of maintenance will be 
minimized when parts of the system are easily related to the application 
(a facet of traceabilty) , manageably small and separately correctable. 
This modularity concept should be incorporated in weapons systems 
software specifications and standards . Furthermore, pieces of the 
problem should be independent. Highly interrelated parts of the problem 
should be in the same piece of the system and unrelated parts should 
reside in unrelated pieces of the system. Finally, implementation and 
maintenance will be minimized when each relationship between pieces of 
the system corresponds only to a relationship between pieces of the 
problem. 

Of great interest for maintenance is the fact that it is very 
difficult to simplify the structure of a program after the program has 
been written. If modularization is undertaken at this point, new 
interfaces must be designed, thus possibly increasing the program’s 
complexity. Also the entire program must be checked to determine which 
parts should be changed and which parts should remain unchanged, as a 
result of modularization. Thus once reduced to code, the structural 
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complexity of a program is essentially fixed; therefore, it is imprac- 
tical to simplify software during the maintenance phase • 

Peters [5] writes that much of the difficulty of software design 
stems from the fact that the problem we are attempting to solve is 
changing while we are solving it. Some of these changes are made by the 
designer as he obtains a better understanding of the problem, and other 
changes come from the user. In either case destabilization of the pro- 
ject and reduction in quality are the results. This situation illus- 
trates the need for software specifications to require a formal change 
procedure . Also, Peters warns agains the fanaticism of some promoters 
of design methods and techniques, pointing out that each method is 
directed at an idealized problem, not necessarily the one to be solved. 
Although there may be some merit to this argument, the use of structured 
design and programming techniques, albeit imperfect, would still achieve 
consistency of documentation and a disciplined approach to design and 
programming . 

Lientz [6] has mentioned that a principle of good design is to 
design with enhancements in mind because, in his survey, it was found 
that most (48 per cent) of maintenance activities stemmed from enhance- 
ments. This, of course, has a direct bearing on maintainability. This 
is a point well made, but one that is difficult to implement in practice 
due to the difficulty of anticipating future operational requirements 
during the design stage. Since many enhancements are the result of 
changing data requirements, it is possible, however, to avoid certain 
practices which are detrimental to maintenance, such as imbedding data 
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descriptions in programs . Hence , a software specification and standard 



should call for independence among program code , data and data bases . 

Heninger and colleagues at the Naval Research Laboratory have 
developed, in conjunction with the A7E aircraft computer system, the 
concept of changes which are likely to occur and those which are not 
likely to occur [7]. A related matter is the identification of functions 
which maintainers would like to modify or remove easily, should the 
need arise. Easy changes should correspond to the most likely changes. 
These considerations evolve into a design principle of separating 
things that will remain the same, no matter what changes are made in 
the rest of the system, from those things that will be affected by the 
changes. System documentation should only specify external behavior 
without implying a particular implementation. For example, programs 
should not have to be changed significantly if changes are made to the 
hardware. In other words, software is described without reference to 
hardware. Another example of the desired immutability of software is 
that software should not change if data arrives in different formats 
over different channels. Thus from the above, a software specification 
and standard should specify that software functions be divided into 
those which will (likely) change and those which will not (unlikely ) 
change . This approach will have the desirable result of causing the 
designers and programmers to think about potential change requests 
early in system development. 
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B. Specification and Documentation Requirements for Achieving Good 



Maintainability 

One of the best approaches for designing a system is to design the 
documentation first. Considerable emphasis should be placed on docu- 
mentation for maintaining software. Hegland suggests determining 
documentation requirements at the beginning of a project [8]. Young 9] 
lists the following types of documentation as being applicable to com- 
puter software: 

Type Project Phase Produced In 



Functional Requirements Document 


Problem Definition 


Data Requirements Document 


Problem Definition 


System and Sub-System Specifications 


System Design 


Program Specification 


System Design 


Data Base Specification 


System Design 


Test Plan 


System Design 


User Manual 


Programming 


Operations Manual 


Programming 


Program Maintenance Manual 


Programming 


Test Analysis Report 


Test 



To the above should be added an Interfaces Specification, which would 
be produced during system design. A software specification and standard 
should require that the documentation to be provided on a project be 
specified . It should also be required that the various levels of 
documentation be consistent (e.g., sub-program specifications should be 
consistent with the associated program specification) . 
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Since good maintainability implies good readability [lo], the 
following aids to readability should be required by the specification 
and standard : 

Comments per line or related lines of code. 

References in the source listings to software specifications, 

and approved changes and test plans. 

References in all documents to related documents. 

Jones [11], based on his study of design and specification techniques, 
concluded it is impossible to conduct a large software project without 
considerable oral communication among project members; it is not 
possible, in his view, to conduct a project mainly on the basis of 
written documentation. One reason for this situation is that written 
documentation may become so bulky that it ceases to be useful. As a 
consequence, according to Jones, word processing costs have become the 
second largest project expense, behind debugging costs. Despite this 
result, we would not propose a reduction in project documentation; on 
the contrary, the thoroughness of documentation should increase in 
quality and quantity. However, the possibility of inundating a project 
with paperwork does suggest the need for a specification and standard to 
stipulate oral communication in the form of design reviews and walk- 
throughs . 

Balzer and Goldman [12] mention understandability as the first 
criterion for judging specifications. Since the specifications are the 
basis of a contract between contractor and customer, they must be clear 
and unambiguous. It seems appropriate , therefore , for a software standard 
to state that a primary objective of the project specifications is to 
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achieve understandability for both the contractor (developer) and cus- 
tomer (user and maintainer) . These authors also make the interesting 
point that the specification itself must be testable. That is, for a 
specification to be considered valid, it is necessary to demonstrate 
that questions (tests) that one might pose concerning proposed system 
operations can be answered satisfactorily by the specifications. Thus, 
a software standard should require that the exercise of “testing” the 
specification for validity be a part of the specification development 
process . This consideration suggests the need for a specification model. 
A carefully designed standard could serve as the model for writing a 
specification. Furthermore , the authors state the importance of making 
the specification independent of the implementation. Thus a specifica- 
tion should state what is to be accomplished and not how it is to be 
implemented. Consequently, another objective of specification develop - 
ment which should be stated in a software standard is the objective of 
achieving independence of the specifications from methods of implemen- 
tation . The current interest in specification languages as a means of 
obtaining consistency and understandability fo specifications has 
implications for maintainability. Balzer and Goldman caution against 
the use of a specification language which optimizes (reduces the resour- 
ces required to execute a program) a system. This would have the 
undesirable effects of reducing understandability, testability and 
maintainability. Thus a software standard should stipulate that 
optimization techniques are not to be employed unless their use is 
unavoidable due to performance requirements . The reason for this is 
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that tricky coding techniques, designed to achieve optimization, 
frequently lead to unmaintainable software. 

Lastly, these authors discuss the effects of specification changes 
on maintainability. They recommend that a specification be so designed 
that changes can be localized and not have an effect on the rest of the 
specification. However, there are instances in which different specifi- 
cations, such as system and program specifications, or different parts 
of a given specification, must be related. Where these relationships 
exist, this fact must be visible in the specification so that changes to 
related specifications can be made when a change is made in a given 
specification. Thus, with regard to the above two points, a software 
standard should stipulate that parts of a specification are to be made 
as independent as possible? but, where relationships must exist, these 
relationships must be made explicit in the system documentation . 

One of the problems with system development is lack of adherence to 
stated software performance objectives over the software's life cycle. 

As stated by McCall [13], little attention is given to identifying the 
qualitiies that the software should exemplify over its life cycle. What 
is needed is a clear statement of performance goals in the user require- 
ments statement, consistency in the use of these goals in subsequent 
stages of development and the ability to trace these goals forward, 
from user requirements phase to maintenance phase; and backward, from 
maintenance to user requirements. The achievement of these objectives 
will be aided if a software specification and standard requires that 
project documentation identify key performance requirements and state 
how the implementation is to satisfy performance requirements . 
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C. Testing Approaches for Achieving Good Maintainability 



DeMillo and colleagues [l4] believe a formal demonstration (test) 
that a program is consistent with its specifications has value only if 
the specifications are derived independently from the program. In 
other words, the specifications should be derived from user require- 
ments and not from the program. Then, if the specification reflects 
user requirements and the program is in accordance with the specifica- 
tion, a demonstration of the program would be meaningful. If the above 
is not the case, the following situation could ensue: A program fails, 

it is changed, and the changes are based on faulty specifications; or 
the specifications are changed, and those changes are based on knowledge 
of the program gained through the failure. In either case, the require- 
ment of using independent criteria is no longer met. To guard against 
this situation, it is necessary for a software standard to specify 
that software design or performance specifications be independently 
derived from user requirements and not, after-the-fact, from the program 
design . Also, as pointed out by Ramamoorthy and colleagues [15] the 
test plan should be independent of the design specification but depen- 
dent upon user requirements. If this is not the case, an error in 
translation from the user requirements to the specification will not be 
caught if the test plan mirrors the specification. The error will be 
caught if the test plan reflects user requirements. In other words, 
software must be tested against what the software was intended to do 
and not against what it is doing. 

A big step towards error reduction in software would result from 
the use of standard and certified reuseable modules [ll]. This practice 
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would obviate the need for module testing of reused modules . Only 
integration testing would be necessary for these modules. The practical- 
ity of this approach is less apparent in embedded computer applications 
than in the commercial applications due to the nonstandard nature of 
the former (e.g., uniqueness of some electronic warfare programs relative 
to accounts payable routines) . However , certain data transformation and 
algorithmic routines may be standard (more or less) . A Software standard 
should require the use of reuseable modules wherever appropriate with 
standard interfaces between modules . 

Design problems are the chief source of programming errors and 
detecting and removing errors are the major programming costs [ll]. 
Therefore, it is of interest to identify the defect removal techniques 
which work best with a given design approach. Jones [ll] lists four 
defect removal methods: 

Design reviews . 

Testing. 

Correctness proofs. 

- Models or prototypes for verifying designs. 

Two major categories of design error are: (1) leaving out needed user 

functions, and (2) putting in functions that users don't need. Correct- 
ness proofs and testing are not guaranteed to find these errors. However, 
methods for depicting models of systems, such as Hierarchy Plus Input- 
Process -Output (HIPO) charts [16 ] and Composite Design [17], in combina- 
tion with design reviews, do provide considerable help in identifying 
missing, unneeded, redundant and ambiguous functions and in specifying 
how the functions should be related and coupled together. These methods 
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have an important ancillary benefit by providing a uniform method of 



system documentation. In view of the above, it is important to not 
rely on one or two methods, such as correctness proofs, to provide 
product quality assurance, but rather to employ a variety of methods 
where the choice of method will depend upon the objective of the test. 
Thus , a software specification and standard should not be restrictive 
in their stipulation of product quality assurance and testing methodolo- 
gies. In fact, it would be worthwhile for these documents to enumerate 
the various methods. (At this time, methods for verifying the design 
against user requirements coupled with formal design reviews are the 
best methods for assuring product quality.) In addition, consideration 
should be given to the adoption of a documentation technique , such as 
HIPO, as a standard for embedded computer development . As noted by 
Pariseau [18] something like HIPO would have improved visibility and 
documentation on the Carrier Based Tactical Support Center project. In 
addition to the use of a documentation standard, one of the most impor- 
tant improvements that could be made in the documentation specification 
is the use of examples, perhaps ones from past projects. Wtjihout 
examples, specifications and standards appear ethereal to the designer 
and programmer . A requirement for a software specification and standard 
to show examples would significantly improve their usability. A related 
consideration is that the choice of documentation technique should not 
be left to the contractor (developer) alone to decide . 

Schneidewind [l9] states that stress or saturation testing is as 
applicable to software as to hardware, particularly with regard to pre- 
mission testing, when it is essential to identify marginal hardware and 
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software (e.g., software which operates satisfactorily under normal load 
but fails — buffer overrun — under peak load) prior to the mission. One 
cannot assume that the software remains invariant between missions, 
because program changes could result from hardware errors (e.g., a 
transient hardware error causes changes in programs residing in memory 
or on disks) , program maintenance activities or errors in revised pro- 
grams which are distributed to the field. 

Other test procedures which will improve maintainability are: 

Static testing (without execution) . 

Dynamic testing (with execution) . 

Regression testing (retest of all modules affected by a software 
change) . 

Testing specifically for the response of the system to unde- 
sirable or unexpected events. This includes stress testing. 
Providing for the repeatability of tests. 

Use of an independent quality control group. 

Thus a software specification and standard should make specific reference 
to the above techniques . 
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V. EVALUATION OF WEAPONS SPECIFICATION WS 8506 



The Fiscal Year 1979 Command and Control System Maintenance Agency 
(CCSMA) System-Level Software Maintenance Approach and Transition Plan 
[20] states that WS 8506 and MIL-STD 1659 will be among the predominant 
factors for software delivered to CCSMA for the Trident ships. Further- 
more , CCSMA is determining what software logic documentation is neces- 
sary for maintenance and where the voids are. The transition plan from 
the development to maintenance phase includes: 

Test and evaluation. 

Validation of documentation. 

Configuration management. 

Introduction of system into the fleet. 

Software maintenance . 

Quality assurance and audits . 

The software delivered to CCSMA by the development agency will 
include : 

Computer program performance specifications. 

Computer program design specifications. 

Interface design specifications. 

Computer program operator ' s manual . 

Test specification requirements. 

Test procedures . 

Thus it is of interest to evaluate WS 8506 in this section and MIL- 
STD 1679 in the next section, particularly with regard to their 
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suitability for maintenance. This evaluation is performed by applying 
the underlined criteria developed in Section IV. Only the criteria which 
are applicable to WS 8506 are utilized in this section. The evaluation 
of both WS 8506 and MIL-STD 1679 will categorize the two documents as 
follows : 

No coverage . 

Inadquate coverage . 

Adequate coverage. 

. - Excellent coverage . 

In addition to the evaluation against the criteria, comments will be 
made about various sections of the documents , where appropriate . 
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TABLE 1 

WS 8506 EVALUATION 



Criterion 



WS 8506 Status 



A. Design Approaches . 

1 . Modularity . 



2. Change procedure. 

3. Independence of code, 
data and data base . 

4. Separation of software 
functions by anticipated 
degree of change. 

B. Specification and Documen- 
tation Requirements . 

1. Specification of 
documentation . 

2. Consistent documentation. 

3. Readability. 

4. Design reviews and 
walkthroughs. 

5. Identification of 
performance require- 
ments . 

C. Test Requirements and 

Relationship to Specification 

Documentation and Design . 

1. Statement of product 
assurance methods. 

2. Documentation standard. 

3. Use of documentation 
examples. 

4. Regression testing. 



Adequate coverage in Sec . 6.2, Com- 
puter Program Design Specification 
Detailed Requirements. 

No coverage . 

No coverage «. 

Adequate coverage in Sec . 5.2, 
Computer Porgram Performance Speci- 
fication Detailed Requirements. 



Excellent coverage throughout. 

Excellent coverage throughout. 

No coverage . 

No coverage . 

Adequate coverage in Sec. 5.2., Com 
puter Program Performance Specifica 
Detailed Requirements. 



Adequate coverage in Sec . 11 , 
Computer Program Test Plan. 

No coverage . 

Excellent coverage throughout. 
No coverage . 
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TABLE 1 (Continued) 



Criterion WS 8506 Status 



5. 


Undesirable and unexpected 


No 


coverage . 




event testing, including 








stress testing. 






6. 


Repeatability of tests. 


No 


coverage . 


7 . 


Independent quality control. 


No 


coverage . 
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